Skip to content

Test: add a repo-level release version consistency test - #297

Open
AmaadMartin wants to merge 1 commit into
mainfrom
feat/release-version-consistency-test
Open

Test: add a repo-level release version consistency test#297
AmaadMartin wants to merge 1 commit into
mainfrom
feat/release-version-consistency-test

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):

N/A

  1. Or, if no issue exists, describe the change:

Problem: release-please-config.json puts the root package, core, dev and integrations in one linked-versions group, so they must always publish the same version. Nothing enforced that. A release that bumps core but skips dev keeps every per-package test green, because each package only checks itself. Only a test that reads across packages catches the drift.

Solution: One integration test reads the four package.json files and .release-please-manifest.json at runtime and asserts they declare one version. It compares them as a single record, so the failure diff names every file that drifted. A second case asserts the packages it checks are exactly the linked-versions group, so a new workspace cannot join the release group and stay unchecked. No version literal appears in the test.

This branch was 52 commits behind main. I rebased it onto 9360bf24 and force-pushed. The diff is one new file.

Collision check: I listed all 666 open PRs on the fork and inspected every version- or release-adjacent one. No other PR touches tests/integration/release/. PRs #258, #417 and #619 add per-package src/version.ts tests under core/test/ and dev/test/. That is a different concern and a disjoint file set.

Note on JSON.parse: the parsed value is typed unknown and narrowed by a type guard. Typing it directly as the expected shape would be an unchecked assertion, which the repository guidelines forbid.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.
npx vitest run --project integration tests/integration/release/version_consistency_test.ts
# Test Files 1 passed (1) | Tests 2 passed (2)
npm run build      # clean
npx eslint tests/integration/release/version_consistency_test.ts     # clean
npx prettier --check tests/integration/release/version_consistency_test.ts
# All matched files use Prettier code style!
npx tsc --noEmit   # zero diagnostics name this file

tsc --noEmit does report pre-existing errors in other test files, for example core/test/a2a/agent_card_test.ts and core/test/utils/live_connection_utils_test.ts. They are on main already and ts:check is not part of the validation workflow, so I left them alone.

The file lands in the integration vitest project, which npm run test:coverage runs. That is the script the validation workflow calls, so the test gates CI.

Proof each test can fail. I mutated the repository files one at a time and confirmed each mutation turns the suite red. I reverted every one with git checkout and re-ran the suite green.

Mutation Failure message
dev/package.json version 1.6.0 -> 1.5.9 record diff: - "dev/package.json": "1.6.0" / + "dev/package.json": "1.5.9"
core/package.json version -> the number 160 record diff: + "core/package.json": 160
All eight version entries -> unreleased expected 'unreleased' to match /^\d+\.\d+\.\d+(?:-[\w.]+)?$/
Fifth package added to the group expected [ '.', 'core', 'dev', ...(2) ] to deeply equal [ '.', 'core', 'dev', 'integrations' ]
packages.dev.component -> dev-tools - "dev": ObjectContaining { "component": "devtools" }
packages.dev -> the bare string "devtools" same ObjectContaining diff
packages -> an array Expected "packages" to be an object in release-please-config.json
plugins -> an object Expected "plugins" to be an array in release-please-config.json
linked-versions plugin removed Expected a "linked-versions" plugin in release-please-config.json
components -> a string Expected "components" to be an array in release-please-config.json
Manifest file -> [] Expected .release-please-manifest.json to hold a JSON object

The third mutation isolates the semver guard: with all eight entries equal, the record comparison still passes and only the guard fails.

Manual End-to-End (E2E) Tests:

npm install
npm run build
npx vitest run --project integration tests/integration/release/version_consistency_test.ts

To watch it fail, change any version field in dev/package.json and re-run.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.

release-please keeps core, dev, integrations and the root package in one
linked-versions group, but nothing enforced that invariant: a partially
applied release could leave the workspaces and
.release-please-manifest.json on different versions with fully green CI.

This test reads the four package.json files, the manifest and
release-please-config.json at runtime and asserts they all declare one
version, and that the set of packages it checks is exactly the
linked-versions group, so adding a workspace to the release group without
extending this test fails loudly.
@AmaadMartin
AmaadMartin force-pushed the feat/release-version-consistency-test branch from 5770a21 to 90d349b Compare August 7, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant